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Abstract 

The infinitesimal transition probability operator for a continuous-time discrete- 
state Markov process, Q, can be decomposed into a symmetric and a skew-symmetric 
parts. As recently shown for the case of diffusion processes, while the symmetric part 
corresponding to a gradient system stands for a reversible Markov process, the skew- 
symmetric part, iu(t) = Au, is mathematically equivalent to a linear Hamiltonian 
dynamics with Hamiltonian H = ^u T (^A T A)2u. It can also be transformed into a 
Schrodinger-like equation = iHu where the "Hamiltonian" operator ~H = —iA is 
Hermitian. In fact, these two representations of a skew-symmetric dynamics emerge 
natually through singular-value and eigen-value decompositions, respectively. The 
stationary probability of the Markov process can be expressed as ||uf || 2 . The motion 
can be viewed as "harmonic" since ^\\u{t) — c|| 2 = where c= (c, c, • • • , c) with c 
being a constant. More interestingly, we discover that 

Tr{A T A) = Y ( ^" w)2 , 

whose right-hand-side is intimately related to the entropy production rate of the Markov 
process in a nonequilibrium steady state with stationary distribution {nj}. The phys- 
ical implication of this intriguing connection between conservative Hamiltonian dy- 
namics and dissipative entropy production remains to be further explored. 



1 Introduction 

Linear operator theory and functional analysis became the central piece of quantum me- 
chanics in the work of Dirac and von Neumann 0] El HI- I n 1930s, Koopman, Birkhoff, 
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von Neumann, and others have also developed a classical dynamical systems theory, includ- 
ing several ergodic theorems, based on linear transformations in Hilbert space [51|5l|6l|7]|. 
One can find this approach to nonlinear dynamical systems in several excellent treatises 
[[8J |9j [TOl [TTJ . In a function space, a Koopman operator maps a function <f>{x) to U t [4>] = 
4>(S t (x)) where S t (x) is the trajectory of an underlying dynamical system. Thus, it rep- 
resents the dynamics in terms of a collection of arbitrary "test functions" defined on a 
moving coordinate system which follows a set of differential equations: "U t [<fi] has at x 
the value which has at the point S t (x) into which x flows after the lapse of the time f ' 
01. This is the deterministic counterpart of Kolmogorov's backward equation; while the 
Perron-Frobenius operator corresponds to Kolmogorov's forward and Liouville equations 

©n. 

Koopman [4] showed that a Hamiltonian dynamics in a certain region of IR 2 " on a 
variety H(q,p) = C of points can be represented in terms of a unitary transformation U t 
in an appropriate Hilbert space: [U t [(p},Ut[ip]) = (4>,tp)- Since U t is a family of one- 
parameter group, it has an infinitesimal generator Q, 

= i<3<f > (x). (1) 

t=o 

Q is self-adjoint, or Hermitian: {Q^tp) = (4>,G^). 

This paper studies the finite-dimensional skew-symmetric linear operator derived from 
decomposition of continuous-time Markov processes lfT2l . It has been shown that for a 
stochastic diffusion process [12], the anti-symmetric part corresponds to a hyperbolic sys- 
tem whose characteristic lines follow a differential equation x = j(x) with V-(p(x)j(x)) = 
0, where p(x) is the stationary density of the diffusion process. Here we show that for a 
continuous-time, discrete-state Markov process, the skew -symmetric part in fact can be 
further mathematically transformed into a Hamiltonian system with a symplectic strucu- 
ture. This last property is the consequence of a skew-symmetric real operator A whose 
eigenvalues are pairs of imaginary numbers. 

Based on the present result for systems with finite dimension, we suspect that an anti- 
symmetric operator A in an appropriate Hilbert space, derived from diffusion process de- 
composition, has a linear, Hamiltonian structure as well. In fact, its Hamiltonian is nothing 
but | ((/>, (A T A) 2 4>) . Note that ((f), A(j)) = 0; and in the dynamics defined by the anti- 
symmetric operator ^4>(t) = A(p, \\4>(t)\\ 2 = (0,0) is a constant of motion. In fact, any 
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operator V that commutes with A, AV = VA, will have 

j t (^V(j>) = {MM) + (<f>,VM) = (0, [-AV + VA]<j>) = 0. (2) 

Mathematically, even for finite dimensional systems, the present analysis is far from 
rigorous or complete; a full treatment remains to be developed. 

2 Decomposition and the skew-symmetric part 
2.1 Dynamics and its different mathematical representations 

When a set of variables changing with time, we say there is a "dynamics". Classical dy- 
namics is customly represented by the time change of the variables themselves, x(t), in 
terms of a system of differential equations 4:x(t) = b(x). In the same vein, stochastic, 
Markov dynamics is represented by dx{t) = b(x)dt + adW(t), first appeared in the work 
of Langevin, now widely known as a stochastic differential equation. 

The work in the 1930s by von Neumann, Koopman, and Birkhoff in USA flUElO, an d 
Kolmogorov, Khinchin, and others in USSR fT3l I71l. however, represents a dynamics by 
a one-parameter family of linear operators in a function space. In the case of Perron- 
Frobenius operator ||8], the corresponding Liouville equation and Kolmogorov forward 
equation are interpreted as the motion of a density function for a collection of particles 
following the classical dynamics. The interpretation of the backward equation, or Koop- 
man operator, on the other hand, is an artitary "test" function in a moving coordinate system 
that follows the classical differential equation. These "modern" mathematical representa- 
tions of dynamics ultimately became the foundation of quantum mechanics and stochastic 
processes. Historically, it is worth pointing out that there was a "direct personal correspon- 
dence between Schrodinger and Kolmogorov at the time" lfl4l . 

One of the insights from these earlier work is that abstract representations of dynamics, 
while might not have simple or intuitive interpretations, can be powerful. In fact, trajec- 
tory, forward, and backward are three different representations of a classical dynamics, 
deterministic or stochastic. While the relation among an ordinary differential equation, 
its Liouville equation and Koopman operator are unambiguously defined, the relation be- 
tween a stochastic differential equation and its forward and backward equations involves 
Ito, Stratonovich, divergence-form, or other interpretations. This has been an important 
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issue in the recent work of P. Ao and his coworkers [fT51 [T6l . It is also noted that many 
studies on entropy productions of diffusion processes had also employed the divergence- 
form elliptic operator [Q/7] [T81IT91 . 

2.2 Decomposition of a continuous-time Markov process 

Traditionally, a continuous-time, discrete-state Markov process (CTDS-MP) is character- 
ized by its infinitesimal transition probability rate matrix, called Q-matrix, in terms of the 
master equation 

j fP (t) = Qp(t), (3) 

in which column vector p(t) = {pi(t)\i = 1, 2, • • • ,n} represents the probability of a 
stochastic system in state i at time t. If the p(t) is a matrix, Eq. [3] is also widely known 
as the Kolmogorov forward equation for the Markov process. It can also be understood in 
terms of a Markov density matrix representation. See Appendix A. 

The stochastic trajectory of a CTDS-MP can be defined through a random time-changed 
Poisson process in terms of multi-variate independent Poisson processes with unit rate [20] . 

Assuming the Markov process is irreducible and recurrent, let it be its unique stationary 
distribution: Qn = 0. We shall denote diagonal matrix II = diag(7r 1; 7r 2 , • • • , n n ). Then Q 
can be decomposed as a forward operator: Q = Qs + Qa with 

Qs = ~ (Q + nQ T n- 1 ) , Q A = l -{Q- UQ^U- 1 ) . (4) 

Q T = Q5 + Qa w1 11 t> e the corresponding decomposition for the backward operator. (See 
Appendix B for a discussion on diffusion process.) 

Note a very important difference between this decomposition for CTDS-MPs and for 
the decomposition for diffusion processes Iil2l : The Qa is no longer a proper Q-matrix; it 
has negative off-diagonal elements. 

The dynamic equation © can be decomposed accordingly: 

ju(t) = (rHgn^ (iH P (*)) = (S + A)u(t) (5) 

in which u(t) = I\~^p{t). It has the dimension of the square root of probability. 

The symmetric part is well understood. It corresponds to a reversible Markov process 
lfT9l with stationary solution = y/Wl. Thus, \ul\ 2 = 7Tj is the stationary probability 
distribution of the Markov process. Extensive studies on the non- symmetric part in term 
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of a circulation decomposition theorem, which characterizes along the path of a process in 
terms of reversible and rotational motions, can be found in [fT9ll2Tll22ll23l , with applica- 
tions to stationary flux analysis in physics and chemistry [|24ll25ll26ll . 
The symmetric part as a n-dimensional linear system 



d 
dt 



u(t) = Su = -V$(u) 



(6) 



has a gradient with potential <£>(«) = — \u T Su. The symmetric matrix (S) is semi- 
positive. More recently, the dynamics of the symmetric part has also been shown as a 
gradient flow in an appropriate Riemannian manifold of probability distributions under a 
Wasserstein metric [|27ll28l . 



2.3 A linear Hamiltonian system 

we now consider the skew-symmetric part as a n-dimensional linear system 



d , . 



(7) 



in which real matrix A is skew-symmetric A T = —A. Eigenvalues of A are pairs of 
imaginary conjugate numbers or zeros, say ±i\i , ±1X2, • ■ • , ±iAfc, 0, • ■ ■ ,0. Therefore, by 
an orthogonal matrix B, B^ 1 = B T , a similarity transformation relates A tcQ 



Ux = BAB 1 



together with 
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(8) 



(9) 



'The orthogonal matrix B is real. This will be shown in the next section. In fact, B ~ V* in Sec. 
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The matrix Hi defines a linear Hamiltonian dynamical system (harmonic oscillator!) 

(10) 



dxj dH dyj dH 



dt dyj ' dt dxj ' 
with Hamiltonian 

1 k 

H(x 1 ,y 2 ,x 2 ,y 2 , ■ ■ ■ , x k , y k , ■ ■ ■ ) = - ^ Xj [x] + y|) . (11) 

3=1 

For the dynamics in ©, the Hamiltonian in (fTTT) is \u T (A T A)^u. To show this, we 
first introduce the notion, 

(A T A)^ = B-\BA T B- l BAB- l )^B 

= B-^nJn^B, (i2) 

i 

in which (HjHi) 2 = E = diag(Ai, A 1; A 2 , A 2 , • • • , A fc , X k , 0, • • • , 0), the singular value 
matrix of A (see Sec. 13.3b . Then following Eq. [9]we have, 

^u T {A T A)^u = H(u). (13) 

Furthermore, one indeed has the conservation 

ju T (A T A)^u = u T (A T (A T A)^ + (A T A^A)u 

= {Buf {nlinln^ + (hIh^Hi) (b u ) 

= (Bu) T {iUlU^ (H\ + Hi)) (Bu) = 0. (14) 
2.4 A Schrodinger-like equation 

Diagonalization of Hi requires working with complex eigenvalues and eigenvectors. Each 
2x2 block in © can be transformed 

Hi -/)US)(-i -0-(o-°0- a* 

Therefor one can denote "Hi = i"H 2 where % is Hermitian. Then A can be written as 
A = B T HiB = iB T H 2 B = iH, where H = B T H 2 B is Hermitian since B is orthogonal. 
The dynamics in © then has another, Schrodinger-like, representation 

iu(t) = iHu. (16) 



The Hamiltonian for (fT6l) is \u T {H 2 ) 1 u. Therefore, H can be legitimately called a Hamil- 
tonian operator. 

From Eq. [131 it is also interesting to note that 

H(u) = -(u T (A T A)*u) = Tr Y,(BpB T ) , (17) 

in which matrix p = uu T . This representation is analogous to that of Heisenberg's in matrix 
mechanics [|29l . 

3 Representations via decompositions of skew-symmetric 
matrix 



We now apply two widely used matrix analysis methods, eigenvalue decomposition (EVD) 
and singular- value decomposition (SVD), to a skew- symmetric matrix [1301 . We shall show 
Schrodinger-equation like and Hamiltonian dynamics natually emerge in these two repre- 
sentations, respectively. 



3.1 2x2 matrix 

First, let us consider skew-symmetric 2x2 matrices in the general form 



It has an eigenvalue decomposition (EVD) (Eq. 



1 
-1 



If 1 -i 
2 I % -1 



i 
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1 -% 
i -1 



iBKB* 



and a singular- value decomposition (SVD): 



1 
-1 



1 

1 



1 
1 



-1 
1 



UZV T . 



(18) 



(19) 



Note that while in general U and V in an SVD are not unique, the U and V in Eq. [T9] are 
unique. 
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3.2 Eigenvalue decomposition (EVD) 

Let skew-symmetric, real matrix A has eigenvalues and corresponding eigenvectors Axg = 
i\x£, where A^ are real and £ = 1, 2, • • • , n. There is at least one A = 0; and for even n, 
there are at least two zero eigenvalues. Furthermore, each iX has a conjugate — iX. As a 
convention, we shall denote X 2 k = — X2k-i < 0. Then 



Ax2k-l — iX2k-lX2k-l, A.X*2k-l — —iX2k-\X2k-l — iX2k%2k-l- 

Therefore, x 2 k = c x*2k~i where c is a complex multiplier. Note that x 2 k-\ and x,2k are 
orthonormal: x 2 k~i • %2k = 0. Hence, x 2 k • %2k = 0, so is x 2 k-i • x 2 k-i = 0. For example, 
these are indeed the case for the column vectors of B and row vectors of B* in Eq. [T8j 



The EVD of A can then be written as 



A = B{iU)B* 
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(20) 



and Xk ■ x e = 5, 



k(- 



The dynamics © in this representation becomes 4v?(t) = iHtp in which (p(t) 



B*u(t). 



3.3 Singular-value decomposition (SVD) 

The SVD of Ogives 
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in which the square diagonal 
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(21) 



and A T Aue = \Xe\ 2 ue with |A 2 fc-i| 
themselves orthogonal matrices. Furthermore, v 2 k-i 
we have 



2 v e . Both U and V are 



| X 2k | . Similarly, A Av £ = \ a? 

u 2k and v 2k = -u 2k -i- Therefore, 
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(22) 



The dynamics © in this representation then becomes 



(23) 



in which = V*u(t). Hi reveals a symplectic structure of the dynamics. "HiS = Hi in 
Eq.gJ 



3.4 Relationships between x and u 

While vector xi are complex, vector u& are real. We have Axt 
Noting \\ k _i = Xl k , we therefore have, 

U 2 k-1 = <^llX 2 k-l + Oii 2 X 2 k, 



iXeXe, A T Axe = \\xi 
(24a) 
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u 2k = a 2 ix 2 k~i + ot 22 x 2k . (24b) 
Accoding to the example in (fT8l) and (fl9"T) . 



an «12 \ / 2 2 



«21 «22 / \ i 1 

2 2 



4 Trace, Hamiltonian, and entropy production 

In terms of the original Markov process with infinitesimal transition rate matrix Q, the 



elements of matrix A = II 2 QIll — III Q T H 2 is: 



= gjgj M! , (25) 



It turns out that the trace of ^4 T ^4 is: 

n n , y z 

Tr(A T A) = 4 = E 



, 1 



(26) 



Therefore, we have 

n n 

Tr(A T A)=J2\\M\ 2 =J2 a l (27) 

The implication of the mathematical equation in (l26l) is very intriguing since according 
to the nonequilibrium steady state (NESS) theory of a Markov process, its entropy produc- 
tion EH EH El is 

e p = E (fti 71 ^ ~ ln ( ^~ 2 - ) • ( 28 ) 

i>j \Qji^i/ 

In particular, when a system is near an equilibrium, q^TTj ~ gjjVTj, then 

1 y (^~^) 2 , (29) 

2 ^ 

1,3=1 ^ 

5 Discussion 

Since 1930s, it has been known that three different types of dynamics, classical determin- 
istic, quantum, and stochastic dynamics, can all be represented in terms of linear operators 
in function space. While Schrodinger, Dirac, von Neumann's quantum mechanics, and 
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Kolmogorov's forward and backward equations for stochastic processes are widely known, 
Koopman's powerful approach to classical dynamics has been mainly limited in mathe- 
matical literature. While Newton and Hamiltonian's classical dynamics are conservative 
for mechanical energy, Fourier's analytical theory of heat, together with the heat equation 
which turns out to the Kolmogorov equation for pure Brownian motion, has been a canon- 
ical example of dissipative systems. 

There is a growing interest in treating stochastic dynamics and statistical thermody- 
namics in a unified framework fl33] |34l [101 135] [36] [3V] |38l [39] gOj- Recently, a decom- 
position of general stochastic diffusion dynamics in function space into symmetric and 
anti-symmetric parts has shown that the former generalizes precisely Fourier's heat equa- 
tion, while the latter generalizes Newtonian conservative dynamics lfT2ll . Furthermore, the 
dynamics decomposition fits perfectly with a recently discovered free energy balance equa- 
tion: The symmetric part has "free energy decreasing = entropy production", as was known 
to Helmholtz and Gibbs, and the anti-symmetric dynamics has free energy conservation. 

A mathematical investigation of anti-symmetric dynamics in a Hilbert space will be 
desirable. In the present work, we seek insights on the anti- symmetric, or skew-symmetric 
dynamics from finite dimensional systems. It is shown that both Hamiltonian representa- 
tion and Schrodinger-like representation natually emergy in the singular- value decomposi- 
tion and eigen-value decomposition of an skew-symmetric matrix. Finally, we discover an 
intriguing connection between the Hamiltonian for the skew-symmetric dynamics and the 
entropy production rate of the original irreversible Markov process. This observation calls 
for re-thinking of the nature of dissipation and time reversibility ffT2ll . 
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Shlizerman, Jin Wang, and Wen- An Yong for helpful discussions. 
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6 Appendices 



6.1 Appendix A: Density matrix for a Markov process 



The solution to Eq. [3] can be formally written as 



p(t) = e Qt p(0). 



(30) 



One can introduce a Markov density matrix: 



p M (t) = e Qt \p(0))(l 



(3D 



in which |p(0)) is an n x 1 matrix, i.e., a column vector, and (1| is a row vector consisting 
of Is. Therefore, Pm(*) is a matrix with Tr(pM) = (1 |_p(t)) = 1. Furthermore, 



Then we have puif) satisfying the same dynamic equation as the Kolmogorov forward 
equation 



with a difference in the initial data p(0): It gives a transition probability matrix if p(0) = I; 
and it gives a density matrix if p M (0) = k>(0)) (1 1 . 

Similarly, a density matrix approach can be formulated for Eq. [16] It yields 



6.2 Appendix B 

The recently introduced "canonical conservative dynamics" lfT2l had been discussed in 01, 
in which the inner product in a Hilbert space is defined with p(x) as a weight |fT9l ; p(x) 
being a positive, single- valued, analytic function on IR n . In the contrary, the weight used 
in lfT2ll is p~ 1 (x). This difference can be seen in the matrix theory: Both I1Q and QII^ 1 
are symmetric for reversible Markov process with generator Q, but only H~^QU^ tran- 
forms the master equation into a gradient system. In fact, Koopman operator and Perron- 
Frobenius operator belong to two different Hilbert spaces with inner products 



pl = e Qt \p(0))(l\=p M . 



(32) 




(33) 



d 

dl 



p{t) = i[Up- pH] ■ 



(34) 




(35) 
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and 

(M)pf = I P~ X {x)<j>{x)^){x)dx, (36) 

J MP 

respectively. Therefore, the symmetric operator in the Koopman (backward) space is 

£*sM = \ (£*[u\ + P~ l £[fM) = V • (A{x)Vu) + (Vlnp)A(x)(Vu), (37) 

while in the Perron-Frobenius (forward) space is [|T2l 

C s [u] = ~ (L[u\+pL* [p^u]) = V-A(x)(Vw-(Vmp)«). (38) 

Indeed, Cs and C* s correspond to the Kolmogorov forward and backward equations of a 
reversible diffusion. 
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Abstract 

The infinitesimal transition probability operator for a continuous-time discrete- 
state Markov process, Q, can be decomposed into a symmetric and a skew-symmetric 
parts. As recently shown for the case of diffusion processes, while the symmetric part 
corresponding to a gradient system stands for a reversible Markov process, the skew- 
symmetric part, iu(t) = Au, is mathematically equivalent to a linear Hamiltonian 
dynamics with Hamiltonian H = ^u T (^A T A)2u. It can also be transformed into a 
Schrodinger-like equation = iHu where the "Hamiltonian" operator ~H = —iA is 
Hermitian. In fact, these two representations of a skew-symmetric dynamics emerge 
natually through singular-value and eigen-value decompositions, respectively. The 
stationary probability of the Markov process can be expressed as ||uf || 2 . The motion 
can be viewed as "harmonic" since ^\\u{t) — c|| 2 = where c= (c, c, • • • , c) with c 
being a constant. More interestingly, we discover that 

Tr{A T A) = Y ( ^" w)2 , 

whose right-hand-side is intimately related to the entropy production rate of the Markov 
process in a nonequilibrium steady state with stationary distribution {nj}. The phys- 
ical implication of this intriguing connection between conservative Hamiltonian dy- 
namics and dissipative entropy production remains to be further explored. 



1 Introduction 

Linear operator theory and functional analysis became the centerpiece of quantum mechan- 
ics in the work of Dirac and von Neumann [jH |2l [3J . In 1930s, Koopman, Birkhoff, von 
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Neumann, and others have also developed a classical dynamical systems theory, includ- 
ing several ergodic theorems, based on linear transformations in Hilbert space [51|5l|6l|7]|. 
One can find this approach to nonlinear dynamical systems in several excellent treatises 
[[8J |9j [TOl [TTJ . In a function space, a Koopman operator maps a function <p{x) to U t [<t>] = 
4>(S t (x)) where S t (x) is the trajectory of an underlying dynamical system. Thus, it rep- 
resents the dynamics in terms of a collection of arbitrary "test functions" defined on a 
moving coordinate system which follows a set of differential equations: "U t [<fi] has at x 
the value which <p has at the point S t (x) into which x flows after the lapse of the time t" 
01. This is the deterministic counterpart of Kolmogorov's backward equation; while the 
Perron-Frobenius operator corresponds to Kolmogorov's forward and Liouville equations 

©n. 

Koopman [4] showed that a Hamiltonian dynamics in a certain region of IR 2 " on a 
variety H(q,p) = C of points can be represented in terms of a unitary transformation U t 
in an appropriate Hilbert space: [U t [(j)},Ut[ip]) = (4>,ip)- Since U t is a family of one- 
parameter group, it has an infinitesimal generator Q, 

= iQ<j>{x). (1) 

t=o 

Q is self-adjoint, or Hermitian: (Q^, ip) = ((f), G^p). 

This paper studies the skew- symmetric linear operator derived from decomposition of 
continuous-time Markov processes [fT2l . It has been shown that for a stochastic diffusion 
process [12], the anti-symmetric part corresponds to a hyperbolic system whose character- 
istic lines follow a differential equation x = j(x) with V • (p(x)j(x)} = 0, where p(x) is 
the stationary density of the diffusion process. Here we show that, as a finite-dimensional 
analogue of the Q, the skew- symmetric part of a continuous-time, discrete-state Markov 
process in fact can be further mathematically transformed into a Hamiltonian system with 
a symplectic strucuture. This is the consequence of a skew-symmetric real operator A 
whose eigenvalues are pairs of imaginary numbers. 

Based on the present result for systems with finite dimension, we suspect that an 
anti-symmetric operator A in an appropriate Hilbert space, derived from diffusion pro- 
cess decomposition, has a linear, Hamiltonian structure as well, with its Hamiltonian be- 
ing |(0, (A T A) 2 (ft). Note that (cf),A(j)) = 0; and in the dynamics defined by the anti- 
symmetric operator 4f(f>{t) = A(p, \\4>(t)\\ 2 = (0,0) is a constant of motion. In fact, any 
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operator V that commutes with A, AV = VA, will have 

j t (0, Vcf>) = (A4>, V<f>) + (0, VM) = (0, [-AV + VA\<t>) = 0. (2) 

Mathematically, even for finite dimensional systems, the present analysis is far from 
rigorous or complete; a full treatment remains to be developed. 

2 Decomposition and the skew-symmetric part 
2.1 Dynamics and its different mathematical representations 

When a set of variables changing with time, we say there is a "dynamics". Classical dy- 
namics is customly represented by the time change of the variables themselves, x(t), in 
terms of a system of differential equations 4:x(t) = b(x). In the same vein, stochastic, 
Markov dynamics is represented by dx(t) = b(x)dt + adW(t), first appeared in the work 
of Langevin, now widely known as a stochastic differential equation. 

The work in the 1930s by von Neumann, Koopman, and Birkhoff in USA flUElO, an d 
Kolmogorov, Khinchin, and others in USSR fT3l I71l. however, represents a dynamics by 
a one-parameter family of linear operators in a function space. In the case of Perron- 
Frobenius operator ||8], the corresponding Liouville equation and Kolmogorov forward 
equation are interpreted as the motion of a density function for a collection of particles 
following the classical dynamics. The interpretation of the backward equation, or Koop- 
man operator, on the other hand, is an artitary "test" function in a moving coordinate system 
that follows the classical differential equation. These "modern" mathematical representa- 
tions of dynamics ultimately became the foundation of quantum mechanics and stochastic 
processes. Historically, it is worth pointing out that there was a "direct personal correspon- 
dence between Schrodinger and Kolmogorov at the time" lfl4l . 

One of the insights from these earlier work is that abstract representations of dynamics, 
while might not have simple or intuitive interpretations, can be powerful. In fact, trajec- 
tory, forward, and backward are three different representations of a classical dynamics, 
deterministic or stochastic. While the relation among an ordinary differential equation, 
its Liouville equation and Koopman operator are unambiguously defined, the relation be- 
tween a stochastic differential equation and its forward and backward equations involves 
Ito, Stratonovich, divergence-form, or other interpretations. This has been an important 
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issue in the recent work of P. Ao and his coworkers [fT51 [T6l . It is also noted that many 
studies on entropy productions of diffusion processes had also employed the divergence- 
form elliptic operator [Q/7] [T81IT91 . 

2.2 Decomposition of a continuous-time Markov process 

Traditionally, a continuous-time, discrete-state Markov process (CTDS-MP) is character- 
ized by its infinitesimal transition probability rate matrix, called Q-matrix, in terms of the 
master equation 

j t p(t) = Qp(t), (3) 

in which column vector p{t) = {pi(t)\i = 1,2, • • • ,n} represents the probability of a 
stochastic system in state i at time t. If the p(t) is a matrix, Eq. [3] is also widely known 
as the Kolmogorov forward equation for the Markov process. It can also be understood in 
terms of a Markov density matrix representation. See Appendix A. 

The stochastic trajectory of a CTDS-MP can be defined through a random time-changed 
Poisson process in terms of multi-variate independent Poisson processes with unit rate [20] . 

Assuming the Markov process is irreducible and recurrent, let 7r be its unique stationary 
distribution: Qn = 0. We shall denote diagonal matrix II = diag^, 7r 2 , • • • , n n ). Then Q 
can be decomposed as a forward operator: Q = Qs + Qa with 

q s = ~ (Q + ns T n- 1 ) , q a = x - (q - ng^rr 1 ) . (4) 

Q T = Q5 + Qa w1 11 t> e the corresponding decomposition for the backward operator. (See 
Appendix B for a discussion on the diffusion process.) 

Note a very important difference between this decomposition for CTDS-MPs and for 
the decomposition for diffusion processes |[T2l : The Qa is no longer a proper Q-matrix; it 
has negative off-diagonal elements. 

The dynamic equation © can be decomposed accordingly: 

ju(t) = (rHgn^ (iH P (*)) = (S + A)u(t) (5) 

in which u(t) = I\~^p{t). It has the dimension of the square root of probability. 

The symmetric part is well understood. It corresponds to a reversible Markov process 
lfT9l with stationary solution = s/Wl. Thus, \ul\ 2 = 7Tj is the stationary probability 
distribution of the Markov process. Extensive studies on the non- symmetric part in term 
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of a circulation decomposition theorem, which characterizes along the path of a process in 
terms of reversible and rotational motions, can be found in [fT9ll2Tll22ll23l , with applica- 
tions to stationary flux analysis in physics and chemistry [|24ll25ll26ll . 
The symmetric part as a n-dimensional linear system 



d 
~dt 



u(t) = Su = -V$(u) 



(6) 



has a gradient with potential <£>(«) = — \u T Su. The symmetric matrix (S) is semi- 
positive. More recently, the dynamics of the symmetric part has also been shown as a 
gradient flow in an appropriate Riemannian manifold of probability distributions under a 
Wasserstein metric [|27ll28l . 



2.3 A linear Hamiltonian system 

we now consider the skew-symmetric part as a n-dimensional linear system 

Au, 



d , . 



(7) 



in which real matrix A is skew-symmetric A T = —A. Eigenvalues of A are pairs of 
imaginary conjugate numbers or zeros, say ±£Ai, ±i\ 2 , • ■ ■ , ±zAfc, 0, • ■ ■ ,0. Therefore, by 
an orthogonal matrix B, B^ 1 = B T , a similarity transformation relates A tcQ 



Ux = BAB 1 






A a 




















o \ 
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-A 2 
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— Afe 
























































































0/ 



together with 



^)B T = (x 1 ,y 1 ,x 2 ,y 2 , ■ ■ ■ ,x k ,y k , ■ ■ ■) 



(8) 



(9) 



'The orthogonal matrix B is real. This will be shown in the next section. In fact, B ~ V* in Sec. 
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Note that the matrix transformation reveals canonical pairs of variables (xj, yA which are 
hidden under the original representation with it's. The matrix Hi defines a linear Hamilto- 
nian dynamical system (harmonic oscillator!) 



dxj dH dyj dH 



(10) 



dt dyj ' dt dxj ' 
with Hamiltonian 

1 k 

H(xi, 2/2, x 2 , y 2 , ■ ■ ■ , x k , y k , ■ ■ ■ ) = - Xj [x] + yf) . (11) 

i=i 

For the dynamics in ©, the Hamiltonian in (fTTT) is ^u T (A T A)^u, where we introduce 
the notion, 

(A T A)^ = B-\BA T B- l BAB- l )^B 

= B- x {U T iUi)^B. (12) 

i 

In Eq. d2l (HJHi) 2 = £ = diag(Ai, Aj, A 2 , A 2 , • • • , A&, A&, 0, • • • , 0) is the singular value 
matrix of A (see Sec. 13.31) . Then following Eq. [9]we have, 

^ T (i T i)5« = #(«). (13) 

Furthermore, one indeed has the conservation 

ju T (A T A)^u = u T (A T (A T A)^ + (A T A)^A)u 

= (Buf (ul{U[Uifz + (HjUi^Ui^ (Bu) 

= (Bu) T ((H^Hi)^ (Hj + Hi)) (Bu) = 0. (14) 

2.4 A Schrodinger-like equation 

Diagonalization of Hi requires working with complex eigenvalues and eigenvectors. Each 
2x2 block in ((8]) can be transformed 

Hi -/)US)(-i -0-(o-°0- <i5) 

Therefor one can denote "Hx = %H 2 where H 2 is Hermitian. Then A can be written as 
A = B T HiB = %B T H 2 B = iH, where H = B T/ H 2 B is Hermitian since B is orthogonal. 
The dynamics in © then has another, Schrodinger-like, representation 

^-u(t) = iHu. (16) 



The Hamiltonian for (|40l) is \u T {H 2 ) 1 u. Therefore, H can be legitimately called a Hamil- 
tonian operator. 

From Eq. [131 it is also interesting to note that 



H(u) = ^(u T (A T A)*v) = Tr^(BpB T ) 



(17) 



in which matrix p = uu T . This representation is analogous to that of Heisenberg's in matrix 
mechanics [291. 



3 Representations via decompositions of skew-symmetric 
matrix 

We now apply two widely used matrix analysis methods, eigenvalue decomposition (EVD) 
and singular- value decomposition (SVD), to a skew- symmetric matrix [1301 . We shall show 
Schrodinger-equation like and Hamiltonian dynamics natually emerge in these two repre- 
sentations, respectively. 

3.1 2x2 matrix 

First, let us consider skew-symmetric 2x2 matrices in the general form 

It has an eigenvalue decomposition (EVD) (Eq. 

1 \ _ 1 / 1 -i \ f i \ / 1 -i 
-1 J ~ 2 V i -1 j V° - % )\ % - 1 
and a singular- value decomposition (SVD): 

o i\ { o o \ f -i o \ UEyT _ (19) 



iBAB*, (18) 



ioy \1 J \0 1 J \ 1 

Note that U and V in an SVD are not unique in general. However, for the 2x2 problem, 
the general forms for orthogonal U and V are 

cos# sin# \ T ^ T ( cos (J) sine 



U =\ -a a > V - ■ i \) (20) 

y — sm cos 6 J y — sm cp cos cp J 

with 9 + = |. Hence, UV T always gives the left-hand- side of Eq. [T9l Furthermore, 

V T U = UV T and VU T = U T V = -UV T . 
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3.2 Eigenvalue decomposition (EVD) 

Let skew-symmetric, real matrix A has eigenvalues and corresponding eigenvectors Axg = 
i\x£, where Xi are real and t = 1, 2, • • • , n. There is at least one A = 0; and for even n, 
there are at least two zero eigenvalues. Furthermore, each i\ has a conjugate — i\. As a 
convention, we shall denote \ 2 k = —\2k-1 < 0. Then 



Ax,2k-\ — iMk-lX2k-l, Ax,2k-\ — —i^2k-\%2k-\ — ^2k^2k-\- 

Therefore, x 2 k = c x 2 k-\ where c is a complex multiplier. Note that x,2k-\ and x,2k are 
orthonormal: X2k-\ • x 2 t = 0. Hence, x 2 k • x 2 k = 0, so is x 2 k-i • %2k-i = 0. For example, 
these are indeed the case for the column vectors of B and row vectors of B* in Eq. [T8j 



The EVD of A can then be written as 



A = B{iH)B* 
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0/ 



(21) 



and x k ■ X£ = 5, 



k(- 



The dynamics © in this representation becomes 4</?(t) = iHtp in which (p(t) 



B*u(t). 



3.3 Singular-value decomposition (SVD) 

The SVD of Ogives 

'111 1 W 

A— U\ u 2 U3 ■ ■ ■ u n 



\ 



E 



v 2 
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in which the square diagonal 
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\X e \ 2 u e with |A 2fc - 




\X 2 k\- 


Similarly, A T Ave = 


\M 2 



(22) 



and A T Aue 

themselves orthogonal matrices. Furthermore, v 2 k-i 
we have 



u 2k and v 2k = -u 2k -i- Therefore, 
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Hi. 



(23) 



The dynamics © in this representation then becomes 



(24) 



in which ^(t) = V*u(t). Hi reveals a symplectic structure of the dynamics. "HiS = Hi in 
Eq.gJ 



3.4 Relationships between x and u 

While vector xi are complex, vector U£ are real. We have Axe 
Noting \^ k _i = Xl k , we therefore have, 



iXeXe, A T Axi = X 



U 2 k-1 — <^l\X 2 k-l + «12^2fc, 



(25a) 
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u 2k = a 2 ix 2 fe_i + a 22 x 2 k- (25b) 
Accoding to the example in (fT8l) and (fl9"T) . 



Q<11 «12 \ / 2 2 



«21 «22 J \ I i 

2 2 



4 Trace, Hamiltonian, and entropy production 

In terms of the original Markov process with infinitesimal transition rate matrix Q, the 



elements of matrix A = II 2 QIll — III Q T H 2 is: 



= ^ (26) 
v /7f^r- 

It turns out that the trace of ^4 T ^4 is: 

Tr(^) = ( '" J ^-' r ' ) - (27) 

Therefore, we have 

n n 

Tr(A T A)=J2\\M\ 2 =J2 a l ( 28 ) 

1=1 i,j=l 

The implication of the mathematical equation in (1271) is very intriguing since according 
to the nonequilibrium steady state (NESS) theory of a Markov process, its entropy produc- 
tion EH EH El is 

e p = ^2 (lijKj ~ Qji^i) In ( J • (29) 

i>j KQjiKi/ 

In particular, when a system is near an equilibrium, q^iTj ~ <2ji7r;, then 

ly fe!^W. (30) 

5 Discussion 

Since 1930s, it has been known that three different types of dynamics, classical determin- 
istic, quantum, and stochastic dynamics, can all be represented in terms of linear operators 
in function space. While Schrodinger, Dirac, von Neumann's quantum mechanics, and 
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Kolmogorov's forward and backward equations for stochastic processes are widely known, 
Koopman's powerful approach to classical dynamics has been mainly limited in mathe- 
matical literature. While Newton and Hamiltonian's classical dynamics are conservative 
for mechanical energy, Fourier's analytical theory of heat, together with the heat equation 
which turns out to the Kolmogorov equation for pure Brownian motion, has been a canon- 
ical example of dissipative systems. 

There is a growing interest in treating stochastic dynamics and statistical thermody- 
namics in a unified framework fl33] |34l [101 135] [36] [3V] |38l [39] gOj- Recently, a decom- 
position of general stochastic diffusion dynamics in function space into symmetric and 
anti-symmetric parts has shown that the former generalizes precisely Fourier's heat equa- 
tion, while the latter generalizes Newtonian conservative dynamics lfT2ll . Furthermore, the 
dynamics decomposition fits perfectly with a recently discovered free energy balance equa- 
tion: The symmetric part has "free energy decreasing = entropy production", as was known 
to Helmholtz and Gibbs, and the anti-symmetric dynamics has free energy conservation. 

A mathematical investigation of anti-symmetric dynamics in a Hilbert space will be 
desirable. In the present work, we seek insights on the anti- symmetric, or skew-symmetric 
dynamics from finite dimensional systems. It is shown that both Hamiltonian representa- 
tion and Schrodinger-like representation natually emergy in the singular- value decomposi- 
tion and eigen-value decomposition of an skew-symmetric matrix. Finally, we discover an 
intriguing connection between the Hamiltonian for the skew-symmetric dynamics and the 
entropy production rate of the original irreversible Markov process. This observation calls 
for re-thinking of the nature of dissipation and time reversibility ffT2ll . 
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6 Appendices 

6.1 Appendix A: Density matrix for a Markov process 

The solution to Eq. [3] can be formally written as 

pit) = e Qt p(0). (31) 

One can introduce a Markov density matrix: 

PM (t) = e Qt \p(0))(l\ (32) 

in which \p(0)) is an n x 1 matrix, i.e., a column vector, and (1| is a row vector consisting 
of Is. Therefore, pM{t) is a matrix with Tr(pM) = (Mp{t)} — 1- Furthermore, 

p 2 M = e Qt \p(0))(l\ =p M . (33) 

Then we have Pm{^) satisfying the same dynamic equation as the Kolmogorov forward 
equation 

>) = QP, (34) 

with a difference in the initial data p(0): It gives a transition probability matrix if p(0) = I; 
and it gives a density matrix if p M (0) = p(0)) (1 1 . 

Similarly, a density matrix approach can be formulated for Eq. [40j It yields 

j t p{t)=i[Hp-pH]. (35) 

6.2 Appendix B 

The recently introduced "canonical conservative dynamics" lfT2l had been discussed in 01, 
in which the inner product in a Hilbert space is defined with p{x) as a weight |fT9l ; p(x) 
being a positive, single- valued, analytic function on IR n . In the contrary, the weight used 
in lfT2l is p~ l (x). This difference can be seen in the matrix theory: Both I1Q and QII^ 1 
are symmetric for reversible Markov process with generator Q, but only H~^QU^ tran- 
forms the master equation into a gradient system. In fact, Koopman operator and Perron- 
Frobenius operator belong to two different Hilbert spaces with inner products 

{ ( P,i J ) K = p(x)<j>(x)if)(x)dx, (36) 



15 



and 



PF 



p 1 (x)(f)(x)ip(x)dx, 



(37) 



respectively. Therefore, the symmetric operator in the Koopman (backward) space is 



C* s [u] = - (£>] + p~ l C[pu\) = V • (A(x)Vu) + (Vlnp)A(x)(Vu), (38) 

while in the Perron-Frobenius (forward) space is [|T2l 

C s [u] = ~ (£[u] + p£* [p _1 «]) = V- A{x)(Vu- (Vlnp)u). (39) 

Indeed, C s and C* s correspond to the Kolmogorov forward and backward equations of a 
reversible diffusion. 

6.3 Appendix C: Examples of skew-symmetric dynamics 

Simple continous skew-symmetric operator: Schrddinger equation. Let us consider the 
Schrodinger equation in free space without potential: 

dip{x, t) 



dt 



with initial value 



(2tto*) : 



(40) 



(41) 



The solution is 

iP(x,t) = (8na 2 y 

Then, 

\mx,t)\\ 2 = 



47T(Jq + i2nht/m 



exp 



x 



4<To + i2ht/m 



(42) 



(Aa 2 y + (2ht/m)- 



exp 



8a 2 x 2 
(Aa 2 ) 2 + (2ht/my 



(43) 



This is a normalized Gaussian distribution with variance 



1 ; \2maJ 



(44) 



The variance is not growing with t, but t 2 : This is not diffusion. The "apparent velocity" 
is ±h/ (2m<7o). The factor cr means Heisenberg's uncertainty principle: If the initial data 
has an "accuracy" of a , then the uncertainty in the velocity is h/a . 
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The solution to Eqn. (|40b has a Fouier representation in time: 



+ cos a/ 2mu / h x + (3 2 sin a/ 2mw / h xj e luJt j do; 

/ gmqj „ _ / 

Kie V— 1 + /3 ie -V 



2^7 i2muj 2mu 

« + a 2 cos \/ — — x + p 2 sin \ / — : — x 



h 



h 



cos cut 



+ % 



s^r i2moj . 2mu 

« — a 2 cos \/ — - — x — p% srn \ / — ; — x 



h 



h 



(45) 



Therefore, if if)(x, 0) is an even, real- valued function of xQ we keep only the 0:2(0;) term: 

2ma> 



Then 



i/j(x,t) 



(x,t)\\'' 



I /"OO 

/ e-^cos 



00 



h 



x ( cos tut — i sin o;t ) do;. 



(46) 



87T(7, 



1 /*O0 /»oo 

^ / / e-o(- 2 ^' 2 ) C os 



2mu 



-x cos 



2mu' 



h V fi 

cos o;t cos o/t + sin o;t sin o/t j da; da;'. 
A discrete skew -symmetric dynamics. Consider anti- symmetric dynamics 

dx 



-x 



dt 



Ax 



(47) 



(48) 



with 

A = 

Then the solution to equation is 

x(t) = e At x(0) in which e At 



( 





-Ai 


\ 








Ai 






















-A 2 








V 








A 2 j 










/ 


cos Ait 


— sin Ait 








\ 






sin Ait 


cos Ait 




















cos A 2 t 


— sin A 2 t 






V 








sin A 2 t 


cos A 2 t 


/ 



(49) 



(50) 



2 More precisely, we only know the \\ip(x, 0)|| 2 , but this does not uniquely specify the ip(x, 0). Thererfore, 
there is a uncertainty in the initial value for ip(x, t). Different choices here lead to different behaviour in the 
following dynamics for t > 0. 
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Therefore, if x(0) = (4=, 0, o) T , then 



However, if x(0) 



1 1 i 

2> 2' ^2' 



/ IklWII 2 \ 
X 2 (t)|| 2 

^3(t)|| 2 

\ IN 

T 



,then 



11^2 
11^3 
V 11^4 



2 \ 

2 

2 
2 



/ cos 2 Ait \ 
sin 2 Ait 
cos 2 A 2 t 
\ sin 2 A 2 t J 



/ l-sin2Ait \ 
/ 4 \ 

l+sin2Aif 
4 

cos 2 A2t 
2 

sin 2 A2t 



V 



/ 



(51) 



(52) 



If Ai and A 2 are non-commensurate, then the x(t) is not periodic. If dimension of A is odd, 
then it has a zero eigenvalue. 

The finite dimensional Eqn. (|52|) should be compared with the infinite dimensional Eqn. 
(|47T) . Heisenberg's uncertainty in the continuous dynamics is a consequence of infinite 
number of non-commensurate frequencies. 
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